Moving picture encoding apparatus

ABSTRACT

Data of both MPEG-2 and MPEG-4 is generated simultaneously with a small circuit scale and a small power consumption. A moving picture encoding apparatus for encoding a moving picture through motion-compensated inter-frame prediction has: a MPEG-2 encoding unit including a motion vector estimator, a frame memory, a forward prediction circuit, a bidirectional prediction circuit, a prediction selection circuit, an intra-frame encoding circuit and a local decoding circuit; a MPEG-4 encoding unit including a frame extraction circuit for extracting a predetermined MPEG-2 frame and a transcoder for encoding the extracted frame; a motion vector calculator calculating a motion vector to be used for MPEG-4 prediction from a motion vector to be used for MPEG-2 prediction; and a prediction mode controller controlling the prediction mode of the MPEG-2 encoding unit in such that the MPEG-2 prediction mode becomes coincident with the MPEG-4 prediction mode.

BACKGROUND OF THE INVENTION

[0001] The present invention relates to a moving picture encodingapparatus for encoding a moving picture by utilizing motion-compensatedinter-frame prediction and to a moving picture recording/reproducingapparatus for recording and reproducing encoded data.

[0002] Technique of converting data encoded by MPEG-2 into encoded dataof MPEG-4 and the like having a different frame rate and a differentimage size is introduced into IEEE TRANSACTIONS ON MULTIMEDIA, Vol. 2,No. 2, JUNE 2000, pp. 101 to 110.

SUMMARY OF THE INVENTION

[0003] This conventional technique adopts a method of decoding data byMPEG-2 and then encoding it by MPEG-4, and is associated with someproblems of a large circuit scale or a large consumption power becauseof a large computation amount.

[0004] Although the conventional technique discloses the conversion intoencoded data having a different frame rate and a different image size,it does not teach the capability of converting data with a small circuitscale or a small consumption power by reducing the computation amount.It neither teaches the simultaneous generation of encoded data having adifferent frame rate and a different image size.

[0005] It is an object of the invention to provide a moving pictureencoding apparatus and a moving picture recording/reproducing apparatuscapable of converting or generating at the same time encoded data havinga different frame rate and a different picture size with a smallcomputation amount, i.e., with a small circuit scale or a smallconsumption power.

[0006] In order to achieve the above object, a moving picture encodingapparatus for encoding a moving picture by utilizing motion-compensatedinter-frame prediction, comprises: a first encoding module for encodingthe moving picture at a first frame rate and at a first image size; asecond encoding module for encoding the moving picture at a second framerate and at a second image size; a prediction mode control module forcontrolling to make a prediction mode of the first encoding module becoincident with a prediction mode of the second encoding module; and amotion vector calculation module for calculating a motion vector to beused by the second encoding module from a motion vector to be used bythe first encoding module.

BRIEF DESCRIPTION OF THE DRAWINGS

[0007] These and other features, objects and advantages of the presentinvention will become more apparent from the following description whentaken in conjunction with the accompanying drawings wherein:

[0008]FIG. 1 is a block diagram showing a moving picture encodingapparatus according to a first embodiment of the invention.

[0009]FIG. 2 is a block diagram showing a moving picture encodingapparatus with an improved shared structure of MPEG-2 and MPEG-4 used bythe embodiment shown in FIG. 1.

[0010]FIG. 3 is a conceptual diagram showing encoded data having adifferent frame rate and a different image size generated at the sametime by a moving picture encoding apparatus according to the invention.

[0011]FIG. 4 is a detailed block diagram showing a prediction selectionunit of the moving picture encoding apparatus according to theinvention.

[0012]FIG. 5 is a diagram illustrating an example of the state ofintra/inter of a macro block before and after transcoding according tothe embodiment shown in FIG. 4.

[0013]FIG. 6 is a detailed block diagram of a prediction selectioncircuit portion of a moving picture. encoding apparatus according to asecond embodiment of the invention.

[0014]FIG. 7 is a diagram illustrating an example of the state ofintra/inter of a macro block before and after transcoding of theembodiment shown in FIG. 6.

[0015]FIG. 8 is a detailed block diagram of a transcoding circuitportion of a moving picture encoding apparatus according to a thirdembodiment of the invention.

[0016]FIG. 9 is a diagram showing the state of motion vector conversionand difference value replacement.

[0017]FIG. 10 is a block diagram showing a moving picturerecording/reproducing apparatus according to an embodiment of theinvention.

[0018]FIG. 11 is a conceptual diagram showing encoded data having adifferent frame rate and a different image size to be converted by themoving picture recording/reproducing apparatus of the embodiment.

[0019]FIG. 12 is a detailed block diagram showing a prediction selectioncircuit of the moving picture recording/reproducing apparatus of theembodiment.

[0020]FIG. 13 is a detailed block diagram showing a transcoder circuitof a moving picture recording/reproducing apparatus according to anotherembodiment of the invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] Embodiments of a moving picture encoding apparatus according tothe invention will be described with reference to the accompanyingdrawings. FIG. 1 is a block diagram showing the outline structure of amoving picture encoding apparatus according to the invention. FIG. 2 isa block diagram showing the outline structure of a moving pictureencoding apparatus with an improved shared structure of MPEG-2 andMPEG-4 used in the apparatus shown in FIG. 1. FIG. 3 is a conceptualdiagram illustrating encoded data having a different frame rate and adifferent image size generated at the same time by a moving pictureencoding apparatus of the invention.

[0022] The structure of the moving picture encoding apparatus shown inFIG. 1 will be described. The moving picture encoding apparatus isconstituted of: a first encoding module (MPEG-2) having a motion vectorestimator 11, a frame memory 12-1, a forward predication circuit 13-1, abidirectional prediction circuit 14-1, a prediction selection circuit15-1, a prediction mode control circuit 16, an intra-frame encodercircuit 17-1 and a local decoder circuit 18-1; a second encoding module(MPEG-4) having a motion vector calculator 21, a frame memory 12-2, aforward predication circuit 13-2, a bidirectional prediction circuit14-2, a prediction selection circuit 15-2, a prediction mode controlcircuit 16, an intra-frame encoder circuit 17-2, a local decoder circuit18-2 and a size conversion circuit 25; a moving picture input terminalT1: an output terminal T2 for high-definition encoded data (MPEG-2); andan output terminal T3 for low-definition encoded data (MPEG-4). Theprediction mode control circuit 16 is used in common by the first andsecond encoding modules.

[0023] The motion vector estimator 11 obtains forward or bidirectionalmotion vector by performing block matching between moving picture data Finput to the moving picture input terminal T1 and a past or futurereference frame Fr stored in the frame memory 12-1.

[0024] The frame memory 12-1 stores a past or future reference frame Fr.

[0025] The forward prediction circuit 13-1 performs forward predictionthrough motion compensation between the input moving picture data F andthe reference frame Fr in accordance with the motion vector to therebygenerate difference data ΔP.

[0026] The bidirectional prediction circuit 14-1 performs bidirectionalprediction through motion compensation between the input moving picturedata F and the reference frame Fr in accordance with the motion vectorobtained by the motion vector estimator 11 to thereby generatedifference data ΔB.

[0027] The prediction selection circuit 15-1 selects one of the inputmoving picture data F itself, forward predicted difference data ΔP andbidirectionally predicted difference data ΔB in accordance with aninstruction from the bidirectional mode control circuit 16.

[0028] The prediction mode control circuit 16 controls the predictionmode of the prediction selection circuit 15-1 in such a manner that theencoding prediction mode of the first encoding module becomes coincidentwith the encoding prediction mode of the second encoding module.

[0029] The intra-frame encoder circuit 17-1 encodes data selected by theprediction selection circuit 15-1 by a compression method utilizingintra-frame correlation such as DCT (Discrete Fourier Transform) tooutput high-definition encoded data (MPEG-2).

[0030] The local decoder circuit 18-1 decodes an I or P frame in theencoded data to generate a reference frame Fr to be used for nextprediction.

[0031] The frame memory 12-2 stores a past or future reference frameFr′.

[0032] The forward prediction circuit 13-2 performs forward predictionthrough motion compensation between the an output F′ of the sizeconversion circuit 25 and the reference frame Fr′ in accordance with amotion vector calculated by the vector calculator 21 to thereby generatedifference data ΔP′.

[0033] The bidirectional prediction circuit 14-2 performs bidirectionalprediction through motion compensation between the output F′ of the sizeconversion circuit 25 and the reference frame Fr′ in accordance with themotion vector calculated by the motion vector calculator 21 to therebygenerate difference data ΔB′. The prediction selection circuit 15-2selects one of the input moving picture data F′ itself, forwardpredicted difference data ΔP′ and bidirectionally predicted differencedata ΔB′ in accordance with an instruction from the prediction modecontrol circuit 16.

[0034] The prediction mode control circuit 16 controls the predictionmode of the prediction selection circuit 15-2 in such a manner that theencoding prediction mode of the first encoding module becomes coincidentwith the encoding prediction mode of the second encoding module.

[0035] The intra-frame encoder circuit 17-2 encodes data selected by theprediction selection circuit 15-2 by a compression method utilizingintra-frame correlation such as DCT to output low-definition encodeddata (MPEG-4).

[0036] The local decoder circuit 18-2 decodes an I or P frame in theencoded data to generate a reference frame Fr′ to be used for nextprediction.

[0037] The motion vector calculator 21 calculates a motion vector to beused after conversion by the second encoding module, by using the motionvector for forward prediction obtained by the motion vector estimator11.

[0038] The size conversion circuit 25 reduces the size of the movingpicture data F input from the moving picture input terminal T1 so as tomatch MPEG-4.

[0039] The moving picture input terminal T1 receives the moving picturedata F.

[0040] The high-definition encoded data output terminal T2 is used foroutputting high-definition encoded data (MPEG-2) encoded by theintra-frame encoding circuit 17-1.

[0041] The low-definition encoded data output terminal T3 is used foroutputting low-definition encoded data (MPEG-4) encoded by theintra-frame encoding circuit 17-2.

[0042] The first encoding module constitutes an encoder of MPEG-2,whereas the second encoding module constitutes an encoder of MPEG-4. Theencoder of MPEG-4 is provided with the size conversion circuit. Thisembodiment is characterized in that both of the prediction modes aremade coincident and that the motion vector of MPEG-4 is calculated fromthe motion vector estimated by MPEG-2.

[0043] By making coincident both the prediction modes of the first andsecond encoding modules, it is possible to allow the encoder of MPEG-4to use the motion vector estimated by MPEG-2. It is not necessary forthe encoder of MPEG-4 to estimate the motion vector. The computationamount can therefore be reduced considerably.

[0044] The structure and operation of a moving picture encodingapparatus shown in FIG. 2 will be described in which the apparatus shownin FIG. 2 shares the common circuits used in the first and secondencoding modules of the moving picture encoding apparatus 1. The movingpicture encoding apparatus 1 shown in FIG. 2 is constituted of a motionvector estimator 11, a frame memory 12, a forward prediction circuit 13,a bidirectional prediction circuit 14, a prediction selection circuit15, a prediction mode control circuit 16, an intra-frame encodingcircuit 17, a local decoder circuit 18, a frame extracting circuit 19, atranscoder circuit 20, a motion vector calculator 21, a moving pictureinput terminal T1, an output terminal T2 for high-definition encodeddata (MPEG-2) and an output terminal T3 for low-definition encoded data(MPEG-4).

[0045] The motion vector estimator 11 obtains forward or bidirectionalmotion vector by performing block matching between moving picture data Finput to the moving picture input terminal T1 and a past or futurereference frame Fr stored in the frame memory 12.

[0046] The frame memory 12 stores a past or future reference frame Fr.

[0047] The forward prediction circuit 13 performs forward predictionthrough motion compensation between the input moving picture data F andthe reference frame Fr in accordance with the motion vector to therebygenerate difference data ΔP.

[0048] The bidirectional prediction circuit 14 performs bidirectionalprediction through motion compensation between the input moving picturedata F and the reference frame Fr in accordance with the motion vectorobtained by the motion vector estimator 11 to thereby generatedifference data ΔB.

[0049] The prediction selection circuit 15 selects one of the inputmoving picture data F itself, forward predicted difference data ΔP andbidirectionally predicted difference data ΔB in accordance with aninstruction from the prediction mode control circuit 16. Namely, theprediction selection circuit 15 outputs picture data after motioncompensation.

[0050] The prediction mode control circuit 16 controls the predictionmode of the prediction selection circuit 15 and the frame extractioncircuit 19 in such a manner that the encoding prediction mode of thefirst encoding module becomes coincident with the encoding predictionmode of the second encoding module.

[0051] The intra-frame encoder circuit 17 encodes data selected by theprediction selection circuit 15 by a compression method utilizingintra-frame correlation such as DCT (Discrete Fourier Transform) tooutput high-definition encoded data (MPEG-2).

[0052] The local decoder circuit 18 decodes an I or P frame in theencoded data to generate a reference frame Fr to be used for nextprediction.

[0053] In response to a control signal from the prediction mode controlcircuit 16, the frame extraction circuit 19 extracts image data or DCTcoefficients of the I and P frame in high-definition data underencoding, as shown in the lower row in FIG. 3.

[0054] The transcoder circuit 20 converts encoded data of the I and Pframes into encoded data having a small image size by using the motionvector calculated by the motion vector calculator 21. Namely, thiscircuit changes a resolution. In the transcoder circuit 20, image dataafter motion compensation is subjected to resolution conversion andthereafter encoded by a compression method utilizing inter-framecorrelation such as DCT, or alternatively coefficient data after DCT isdirectly converted to coefficient data after resolution conversion toencode the data.

[0055] The motion vector calculator 21 calculates motion vector fortranscoding at the transcoder circuit 20 by using the motion vector forforward prediction obtained by the motion vector estimator 11.

[0056] The moving picture input terminal T1 receives the moving picturedata F.

[0057] The high-definition encoded data output terminal T2 is used foroutputting high-definition encoded data (MPEG-2) encoded by theintra-frame encoding circuit 17.

[0058] The low-definition encoded data output terminal T3 is used foroutputting low-definition encoded data (MPEG-4) transformed by thetranscoder circuit 20.

[0059] The operation of the moving picture encoding apparatus 1 of theinvention will be described. The motion vector estimator 11 obtainsforward or bidirectional motion vector through block matching betweenmoving picture data F input to the moving picture input terminal T1 andthe past or future reference frame Fr stored in the frame memory 12, andsends it to the forward prediction circuit 13 or bidirectionalprediction circuit 14.

[0060] The forward prediction circuit 13 performs forward predictionthrough motion compensation between the input moving picture data F andthe reference frame Fr in accordance with the motion vector to generatethe difference data ΔP. The bidirectional prediction circuit 14 performsbidirectional prediction through motion compensation between the inputmoving picture data F and the reference frame Fr in accordance with themotion vector to generate the difference data ΔB.

[0061] In response to an instruction from the prediction mode controlcircuit 16, the prediction selection circuit 15 selects one of the inputmoving picture data F itself, forward predicted difference ΔP andbidirectionally predicted difference ΔB and sends the selected one tothe intra-frame encoding circuit 17. The intra-frame encoding circuit 17encodes the selected data by a compression method utilizing intraframecorrelation such as DCT to generate high-definition encoded data(MPEG-2) which is output from the high-definition encoded data outputterminal T2.

[0062] In this embodiment, the control of the prediction mode forencoding (prediction mode) is intended to include both a frame typeselection control for controlling which one of the I/P/B frames is to beselected as a frame to be encoded and a macro block mode selectioncontrol for controlling which one of intra/inter is to be selected foreach macro block. Also in this embodiment, the process starting from theinput terminal T1 to the prediction selection circuit 15 is calledmotion compensation.

[0063] The control of the encoding prediction mode is performed so thatthe pattern such as shown in the middle row of FIG. 3 is obtained. InFIG. 3, each rectangle represents one frame of a moving picture, and theabscissa represents time. A frame indicated by I is an I frame obtainedby directly encoding an input moving picture. A frame indicated by P isa P frame obtained through encoding by forward prediction. A frameindicated by B is a B frame obtained through encoding by bidirectionalprediction. A conceptual example of an encoding process by MPEG-2 andMPEG-4 will be described with reference to FIG. 3. In FIG. 3, the upperrow shows frames of input pictures and affixes indicate a frame number.The middle row shows image data after the encoding process of MPEG-2 andaffixes indicate a corresponding frame number. The lower row indicatesimage data after the encoding process of MPEG-4 and affixes indicate acorresponding frame number. It is assumed that the frames F₀ to F₁₁input to the picture input terminal T1 are rearranged beforehand in theorder of F₂, F₀, F₁, F₅, F₃, F₄, F₈, F₆, F₇, F₁₁, F₉, and F₁₀ in theframe number order shown in the middle row.

[0064] (1) A frame I₂ is encoded by using only the frame F₂.

[0065] (2) A frame B₀ is encoded by using the frame F₀ and a referenceframe (hereinafter the reference frame is indicated by a broken line)Fr₂ decoded from the I₂ frame.

[0066] (3) A frame B₁ is encoded by using the frame F₁ and the referenceframe Fr₂ decoded from the I₂ frame.

[0067] (4) A frame P₅ is encoded by using the frame F₅ and the referenceframe Fr₂ decoded from the I₂ frame.

[0068] (5) A frame B₃ is encoded by using the frame F₃ and the referenceframes Fr₂ and Fr₅ decoded from the I₂ P₅ frames.

[0069] (6) A frame B₄ is encoded by using the frame F₄ and the referenceframes Fr₂ and Fr₅ decoded from the I₂ P₅ frames.

[0070] (7) A frame P₈ is encoded by using the frame F₈ and the referenceframe Fr₅ decoded from the P₅ frame.

[0071] (8) A frame B₆ is encoded by using the frame F₆ and the referenceframes Fr₅ and F₈ decoded from the P₅ P₈ frames.

[0072] (9) A frame B₇ is encoded by using the frame F₇ and the referenceframes Fr₅ and Fr₈ decoded from the P₅ and P₈ frames.

[0073] (10) A frame P₁₁ is encoded by using the frame F₁₁ and thereference frame Fr₈ decoded from the P₈ frame.

[0074] (11) A frame B₉ is encoded by using the frame F₉ and thereference frames Fr₈ and Fr₁₁ decoded from the P₈ and P₁₁ frames.

[0075] (12) A frame B₁₀ is encoded by using the frame F₁₁ and thereference frames Fr₈ and Fr₁₁ decoded from the P₈ and P₁₁ frames.

[0076] Encoding is performed thereafter in a similar manner by forwardand bidirectional predictions.

[0077] The local decoder circuit 18 decodes the I and P frames in theencoded data and stores it in the frame memory 12 as the reference frameFr to be used for the next prediction.

[0078] Next, the operation of low-definition encoding will be described.In response to a control signal from the prediction control circuit 16,the frame extraction circuit 19 extracts data of the I and P frames fromthe high-definition encoded data under encoding and sends it to thetranscoder circuit 20.

[0079] The transcoder circuit 20 converts the encoded data of the I andP frames into encoded data (MPEG-4) having a smaller image size andoutputs it from the low-definition encoded data output terminal T3. Inthis case, the motion vector calculator 21 calculates the motion vectorafter conversion by using the motion vector for forward predictionobtained by the motion vector estimator 11.

[0080] With the above operations, since the same prediction mode is usedfor the high-definition encoding and low-definition encoding, it is notnecessary to perform motion vector estimation for low-definitionencoding. Since the computation amount can be reduced, both thehigh-definition and low-definition data can be generated at the sametime with a small circuit scale and a small consumption power.

[0081] Next, the control of a macro block mode by the predictionselection circuit will be described with reference to FIG. 4 which is adetailed block diagram of the prediction selection circuit 15. Theprediction selection circuit 15 has a frame mode selection circuit 151,an intra/inter judgement circuit 152 and a macro block mode selectioncircuit 153.

[0082] The frame mode selection circuit 151 selects one of the inputdata F, ΔP and ΔBm in accordance with which one of the frames I/P/B isselected. If the I frame is selected, the input data F is selected. Ifthe P frame is selected, the forward prediction difference ΔP isselected.

[0083] If the B frame is selected, the intra/inter judgement circuit 152judges whether it is more efficient to encode either the moving picturedata F itself or the bidirectional prediction difference ΔB, bothincoming at each micro block which is the encoding unit. In accordancewith the judgement result, the macro block selection circuit 153 selectsthe moving picture data F or difference ΔB for each macro block andoutputs difference data ΔBm selected for each macro block. Thisjudgement for each macro block is not performed for the P frame.

[0084] The state of the intra (encoding of moving picture itself)/inter(encoding of prediction difference value) of a micro block before andafter encoding of each frame will be described with reference to FIG. 5.In the example shown in FIG. 5, an image size is reduced by one/fourthboth in the horizontal and vertical directions to perform encoding. Eachblock represents a macro block. In the I frame, blocks are all intramacro blocks constituting one I frame. In the P frame, blocks are allinter macro blocks constituting one P frame. In the B frame, blocks aremade of mixed intra and inter macro blocks, which poses no problembecause transcoding is not performed. Since the same mode is used for aplurality of macro blocks to be transcoded, it is not necessary toperform intra/inter conversion even if prediction is required to beperformed again. The computation amount can therefore be reduced, andboth the high-definition and low-definition data can be generated at thesame time with a small circuit scale and a small consumption power.

[0085] If the macro block mode of the P frame is performed only by theinter scheme, an encoding efficiency and an image quality may be loweredwhen an abrupt picture change such as a scene change occurs.

[0086] Next, the embodiment dealing with such a case will be describedwith reference to the block diagram of FIG. 6. The circuit portions notshown in FIG. 6 are the same as those of the embodiment shown in FIG. 2.In this embodiment shown in FIG. 6, an image quality deteriorationdetector circuit 22 is added for monitoring an output of the intra-frameencoding circuit 17 and detecting an image quality deterioration, andthe prediction selection circuit 15 is constituted of a frame modeselection circuit 151, an intra/inter judgement circuit 152, a microblock mode selection circuit 153 and a frame unit macro block modeselection circuit 154.

[0087] The image quality deterioration detection circuit 22 receivesinformation representative of quantization coarseness or the like havinga high correlation with the image quality from the intra-frame encodingcircuit 17, monitors this information to detect an image qualitydeterioration of a P frame to be caused by a scene change or the like,and issues a command at the next P frame to the frame unit macro blockmode selection circuit 154.

[0088] The frame unit macro block mode selection circuit 154 selects,during the next frame which received the command, the input movingpicture data F itself (intra micro blocks) in place of the forwardprediction difference data ΔP. Therefore, even if the image quality isdeteriorated by the scene change, the efficiency is improved in the nextP frame with intra macro blocks so that the image quality can berecovered.

[0089]FIG. 7 shows the state of selecting intra/inter macro blocks forthe P frame before and after transcoding according to the embodiment.For the P frame immediately after the image quality deterioration, theframe unit macro block mode selection circuit 154 selects not the intermacro blocks corresponding to the difference ΔP supplied from theforward prediction circuit 13 but the intra macro blocks of the inputpicture data F and performs transcoding of the intra macro blocks. Forother P frames, the circuit 154 selects the inter macro blockscorresponding to the difference ΔP and performs transcoding of the intermacro blocks. Also in this embodiment, since the same mode is used for aplurality of macro blocks to be transcoded, it is not necessary toperform intra/inter conversion even if prediction is required to beperformed again. The computation amount can therefore be reduced, andboth the high-definition and low-definition data can be generated at thesame time with a small circuit scale and a small consumption power.

[0090] Next, another embodiment will be described with reference to FIG.8, in which the motion vector estimator 11 and transcoder circuit 20further reduce the computation amount. FIG. 8 is a block diagram showingthe outline of a low-definition encoding process section of theembodiment. The circuit portions not shown in FIG. 8 are similar tothose of the embodiment shown in FIG. 2. In the embodiment shown in FIG.8, a difference value invalidating circuit 23 and a motion vectorcomparison circuit 24 are additionally used and the motion vectorcalculator 21 is constituted of a motion vector selector 211 and amotion vector scaling circuit 142. In the motion vector calculator 21,the motion vector selector 211 selects an optimum motion vector from aplurality of motion vectors sent from the motion vector estimator 11,and the motion vector scaling circuit 212 converts the scale of theselected optimum motion vector so as to make it match the image sizeafter transcoding.

[0091] The motion vector comparison circuit 24 compares the selectedmotion vector with a plurality of motion vectors, and if there is alarge difference, issues a command to the difference value invalidatingcircuit 23. Upon reception of this command, the difference valueinvalidating circuit 23 replaces the difference value componentscorresponding to the motion vector with a value of 0. This replacementstate is illustrated in FIG. 9. The motion vector selector 211 monitorsmotion vectors in blocks of a frame and selects the motion vector whichis largest in number in the frame. The motion vector scaling circuit 212changes the scale of the motion vector by using the selected motionvector. If the motion vector supplied from the motion vector estimator11 is different from the motion vector selected by the motion vectorselector 211, the vector comparison circuit 24 issues a command to thedifference value invalidating circuit 23 whereat the difference value isreplaced with 0. This image is reduced in size by the transcoder circuit20 so that the image replaced with 0 after transcoding is filled withthe reference image data subjected to motion compensation by the samemotion vector as nearby motion vectors.

[0092] Therefore, although there is some deviation from an originalimage, thee is no image quality deterioration such as large noises to becaused by different difference data. It is therefore unnecessary toperform again motion compensation prediction also for the image portionwith different motion vectors. The computation amount can therefore bereduced, and both the high-definition and low-definition data can begenerated at the same time with a small circuit scale and a smallconsumption power.

[0093] With reference to the block diagram shown in FIG. 10, the outlinestructure and operation of a moving picture recording/reproducingapparatus according to a second embodiment of the invention will bedescribed. This moving picture recording/reproducing apparatus canrecord an input moving picture as high-definition encoded data (MPEG-2)and reproduce both a MPEG-2image and low-definition encoded data(MPEG-4). The structure and operation of a recording apparatusconstituted of the circuit portion from the motion vector estimator 11to the local decoder circuit 18 are the same as those of the embodimentshown in FIG. 2, and so the description thereof is omitted.High-definition encoded data is recorded via a recorder 100 into astorage medium not shown.

[0094] The reproducing apparatus is constituted of a reproducing circuit101, a frame memory 52, a forward prediction circuit 53, a bidirectionalprediction circuit 54, a prediction selection circuit 55, a predictionmode detection circuit 56, an intra-frame decoding circuit 57, a frameextraction circuit 19, a transcoder circuit 20, a motion vectorcalculator 21, a low-definition encoded data output terminal T3, and amoving picture data output terminal T4.

[0095] The reproducing circuit 101 reproduces high-definition encodeddata (MPEG-2) from the storage medium not shown.

[0096] The frame memory 52 stores data (I/P/B frame) selected by theprediction selection circuit 55 as a next reference frame Fr.

[0097] The forward prediction circuit 53 performs forward predictionthrough motion compensation by the reference frame Fr in accordance withthe motion vector contained in the data decoded by the intra-framedecoding circuit 57, and adds the difference value ΔP to recover theoriginal data (P frame).

[0098] The bidirectional prediction circuit 54 performs bidirectionalprediction through motion compensation by the reference frame Fr inaccordance with the motion vector contained in the data decoded by theintra-frame decoding circuit 57, and adds the difference value ΔB torecover the original data (B frame).

[0099] The prediction selection circuit 55 selects one of the decodedmoving picture data (I frame) itself, data (P frame) recovered byforward prediction and data (B frame) recovered by bidirectionalprediction, in accordance with I/P/B information.

[0100] The prediction mode detection circuit 56 detects I/P/Binformation in accordance with prediction mode information decoded bythe intra-frame decoding circuit 57.

[0101] The intra-frame decoding circuit 57 decodes high-definitionencoded data reproduced by the reproducing circuit 101 to generate themoving picture data (I frame) itself, prediction difference value ΔP orΔB. The intra-frame decoding circuit 57 also decodes I/P/B predictionmode information and a motion vector value.

[0102] The frame extraction circuit 19 extracts I or P frame data fromhigh-definition encoded data under decoding as shown in the lower row inFIG. 11, in response to a control signal from the prediction modedetection circuit 56.

[0103] The transcoder circuit 20 converts I and P frame encoded datainto low-definition encoded data (MPEG-4) having a smaller image size.

[0104] The motion vector calculator 21 calculates motion vector afterconversion by using a forward prediction motion vector.

[0105] The low-definition encoded data output terminal T3 is used foroutputting low-definition encoded data (MPEG-4).

[0106] The moving picture data output terminal T4 is used for outputtingdata (I/P/B frame) selected by the prediction selection circuit 55.

[0107] Next, the reproduction operation will be described. Theintra-frame decoding circuit 57 decodes the high-definition encoded data(MPEG-2) reproduced from the storage medium by the reproducing circuit101 to generate the moving picture data (I frame) itself, predictiondifference value ΔP or ΔB. The intra-frame decoding circuit 57 alsodecodes the I/P/B prediction mode information and a motion vector value.

[0108] The forward prediction circuit 53 performs forward predictionthrough motion compensation by the reference frame Fr in accordance withthe motion vector, and adds the difference value ΔP to recover theoriginal data (P frame). The bidirectional prediction circuit 54performs bidirectional prediction through motion compensation by thereference frame Fr in accordance with the motion vector, and adds thedifference value ΔB to recover the original data (B frame).

[0109] The prediction selection circuit 55 selects one of the decodedmoving picture data (I frame) itself, the data (P frame) recovered byforward prediction and the data (B frame) recovered by bidirectionalprediction, in accordance with the I/P/B information detected by theprediction mode detection circuit 56. The selected data is output fromthe moving picture data output terminal T4, and stored in the framememory 52 as the next reference frame.

[0110] Next, conversion into low-definition encoded data will bedescribed with reference to FIG. 11. FIG. 11 is a conceptual diagramshowing encoded data having a different frame rate and a different imagesize to be converted by the moving picture recording/reproducingapparatus of the embodiment. In response to a control signal from theprediction mode detection circuit 56, the frame extraction circuit 19extracts data of the I and P frames from the high-definition encodeddata under decoding and sends it to the transcoder circuit 20. Thetranscoder circuit 20 converts the encoded data of the I and P framesinto encoded data (MPEG-4) having a smaller image size by using motionvector after conversion supplied from the motion vector calculator 21,and outputs it from the low-definition encoded data output terminal T3.In this case, the motion vector calculator 21 calculates the motionvector after conversion by using the motion vector for forwardprediction.

[0111] With the above operations, since the same prediction mode is usedfor the high-definition encoding and low-definition encoding, it is notnecessary to perform vector estimation for low-definition encoding.Since the computation amount can be reduced, the low-definition data canbe converted from the high-definition data, with a small circuit scaleand a small consumption power.

[0112] Next, the control of a macro block mode during the reproductionwill be described with reference to FIG. 12 showing the detailed blockdiagram of the prediction selection circuit 55 of the reproducingapparatus shown in FIG. 10. The prediction selection circuit 55 has aframe mode selection circuit 551, a macro block mode selection circuit552 and a macro block mode selection circuit 553.

[0113] The frame mode selection circuit 551 selects data in accordancewith I/P/B information.

[0114] The macro block mode selection circuits 552 and 553 select the Iframe information, forward prediction data ΔP or bidirectionalprediction data ΔB decoded by the intra-frame decoding circuit 57, inaccordance with intra/inter information which is attribute informationof each macro block. Data output from the macro block mode selectioncircuit 552 is represented by Pm, and data output from the macro modeselection circuit 553 is represented by Bm.

[0115] The frame mode selection circuit 551 selects data in accordancewith I/P/B information. The intraframe decoding circuit 57 decodes theintra/inter information of each macro block, and in accordance with thedecoded information, the macro block mode selection circuits 552 and 553select data.

[0116] The detailed block diagram of the prediction selection circuit 15of the recording apparatus is the same as that shown in FIG. 4. Thestate of intra/inter of decoded macro blocks is similar to that shown inFIG. 5. Since the same mode is used for a plurality of macro blocks tobe transcoded, it is not necessary to perform intra/inter conversioneven if prediction is required to be performed again. The computationamount can therefore be reduced, and the low-definition encoded data canbe converted from the high-definition data, with a small circuit scaleand a small consumption power.

[0117] In FIG. 10, the prediction selection circuit 15 of the recordingapparatus may be changed to the prediction selection circuit 15 shown inthe block diagram of FIG. 6. In this case, similar to the encodingapparatus, even if the image quality is deteriorated by a scene change,the efficiency is improved at the next P frame by intra so that theimage quality is recovered. The state of selection of intra/inter formacro blocks in the P frame is similar to that shown in FIG. 7. Also inthis embodiment, since the same mode is used for a plurality of macroblocks to be transcoded, it is not necessary to perform intra/interconversion even if prediction is required to be performed again. Thecomputation amount can therefore be reduced, and the low-definitionencoded data can be converted from the high-definition data, with asmall circuit scale and a small consumption power.

[0118] Next, another embodiment will be described with reference to FIG.13, in which the motion vector estimator 11 and transcoder circuit 20further reduce the computation amount. The circuit portions not shown inFIG. 13 are similar to those of the embodiment shown in FIG. 10. Similarto the encoding apparatus, a difference value invalidating circuit 23and a motion vector comparison circuit 24 are additionally used. In themotion vector calculator 21, the motion vector selector 211 selects anoptimum motion vector from a plurality of motion vectors sent from theforward prediction circuit 53, and the motion vector scaling circuit 212converts the scale of the selected optimum motion vector so as to makeit match the image size after transcoding.

[0119] The motion vector comparison circuit 24 compares the selectedmotion vector with a plurality of motion vectors, and if there is alarge difference, issues a command to the difference value invalidatingcircuit 23. Upon reception of this command, the difference valueinvalidating circuit 23 replaces the difference value componentscorresponding to-the motion vector with a value of 0. This replacementstate is illustrated in FIG. 9.

[0120] The image replaced with 0 after transcoding is filled with thereference image data subjected to motion compensation by the same motionvector as nearby motion vectors. Therefore, although there is somedeviation from an original image, there is no image qualitydeterioration such as large noises to be caused by different differencedata. It is therefore unnecessary to perform again motion compensationprediction also for the image portion with different motion vectors. Thecomputation amount can therefore be reduced, and both thehigh-definition and low-definition data can be generated at the sametime with a small circuit scale and a small consumption power.

[0121] As described so far in each embodiment, in encoding of MPEG-4,the motion vector estimation is not performed. A half of the consumptionpower is occupied by the motion vector estimation. According to eachembodiment of the invention, the power consumption of MPEG-4 can behalved. For example, in dual encoding of MPEG-2 and MPEG-4, theconsumption power of MPEG-4 is about one fifth of the total powerconsumption. Since the consumption power of MPEG-4 is halved, the totalconsumption power can be reduced by about one tenth.

[0122] According to the invention, it is possible to generate both highand low-definition encoded data at the same time with a small circuitscale and with a small power consumption and to convert high-definitionencoded data into low-definition encoded data.

[0123] While we have shown and described several embodiments inaccordance with our invention, it should by understood that disclosedembodiments are susceptible of changes and modifications withoutdeparting from the scope of the invention. Therefore, we do not intendto be bound by the details shown and described herein but intend tocover all such changes and modifications within the ambit of theappended claims.

We claim:
 1. A moving picture encoding apparatus for encoding a movingpicture by utilizing motion-compensated inter-frame prediction,comprising: a first encoding module for encoding the moving picture at afirst frame rate and at a first image size; a second encoding module forencoding the moving picture at a second frame rate and at a second imagesize; a motion vector calculation module for calculating a motion vectorto be used by said second encoding module from a motion vector to beused by said first encoding module; and a prediction mode control modulefor controlling to make a prediction mode of said first encoding modulebe coincident with a prediction mode of said second encoding module, sothat said motion vector calculated by said motion vector calculationmodule can be used by said second encoding module.
 2. A moving pictureencoding apparatus for encoding a moving picture by utilizingmotion-compensated inter-frame prediction, comprising: a first encodingmodule for encoding the moving picture at a first frame rate and at afirst image size, by performing at least motion compensation and DCT; asecond encoding module for encoding the moving picture at a second framerate and at a second image size, by extracting data after the motioncompensation or DCT by said first encoding module; a motion vectorcalculation module for calculating a motion vector to be used by saidsecond encoding module from a motion vector to be used by said firstencoding module; and a prediction mode control module for controlling tomake a prediction mode of said first encoding module be coincident witha prediction mode of said second encoding module, so that said motionvector calculated by said motion vector calculation module can be usedby said second encoding module.
 3. A moving picture encoding apparatusfor encoding a moving picture by utilizing motion-compensatedinter-frame prediction, comprising: a first encoding module for encodingan input moving picture at a first frame rate and at a first image size;a second encoding module for extracting picture data under encoding bysaid first encoding module and encoding the picture data at a secondframe rate and at a second image size; a motion vector calculationmodule for calculating a motion vector to be used by said secondencoding module from a motion vector to be used by said first encodingmodule; and a prediction mode control module for controlling aprediction mode of said first encoding module so as to make theprediction mode of said first encoding module be coincident with aprediction mode of said second encoding module, so that said motionvector calculated by said motion vector calculation module can be usedby said second encoding module.
 4. A moving picture encoding apparatusaccording to claim 3, wherein said prediction mode control module sets aforward prediction frame to the prediction mode of said first encodingmodule for the prediction mode of said second encoding module relativeto the forward prediction frame using only prediction from past frames.5. A moving picture encoding apparatus according to claim 3, whereinsaid prediction mode control module controls in such a manner that theprediction mode of said first encoding module for each partial picturefor the prediction mode of said second encoding module relative to theforward prediction frame using only prediction from past frames, encodesa difference value from a prediction value for each of all partialpictures in the same frame or encodes each of all partial pictures inthe same frame without using prediction.
 6. A moving picture encodingapparatus according to claim 5, further comprising: image qualitydeterioration detecting module for detecting an image qualitydeterioration of a frame obtained by said first encoding module byencoding difference values from the prediction value for all partialpictures in the same frame, wherein when said image qualitydeterioration detecting module detects an image quality deterioration,said prediction mode control module controls in such a manner that theprediction mode of said first encoding module for each partial picturefor the prediction mode of said second encoding module relative to theforward prediction frame using only prediction from past frames, encodesall partial pictures themselves in the same frame without usingprediction.
 7. A moving picture encoding apparatus according to claim 3,wherein: said motion vector calculation module calculates one motionvector from a plurality of motion vectors given to a plurality ofpartial pictures of said first encoding module, the partial picturescorresponding to one partial picture of said first encoding module; andsaid apparatus further comprises difference value invalidating modulefor, if a motion vector of said first encoding module is greatlydifferent from a motion vector calculated by said motion vectorcalculation module, replacing a difference value of said second encodingmodule corresponding to a partial picture of said first encoding modulehaving such a motion vector with 0 to perform encoding.
 8. A movingpicture recording/reproducing apparatus for recording encoded data of amoving picture obtained through motion-compensated inter-frameprediction in a storage medium and reproducing the encoded and recordeddata from the storage medium and decoding the encoded data, comprising:a first encoding module for encoding an input moving picture at a firstframe rate and at a first image size; a decoding module for decodingdata encoded by said first encoding module; re-encoding module forre-encoding a predetermined frame extracted from frames decoded by saiddecoding module, at a second frame rate smaller than the first framerate and at a second image size smaller than the first image size;motion vector calculation module for calculating a motion vector to beused for prediction by said re-encoding module from a motion vectordecoded by said decoding module; and prediction mode control module forcontrolling a prediction mode of said first encoding module to make theprediction mode of said first encoding module be coincident with aprediction mode of said re-encoding module, so that said motion vectorcalculated by said motion vector calculation module can be used by saidre-encoding module.
 9. A moving picture reproducing apparatus forreproducing data of a moving picture encoded at a first frame rate andat a first image size through motion-compensated inter-frame predictionfrom a storage medium and decoding the encoded data, comprising:decoding module for decoding the encoded data reproduced from thestorage medium; prediction mode detection module for detecting aprediction mode of a frame decoded by said decoding module; motionvector calculation module for calculating a motion vector to be used forprediction by re-encoding module from a motion vector decoded by saiddecoding module; and said re-encoding module for re-encoding a movingpicture decoded by said decoding module at a second frame rate smallerthan the first frame rate and at a second image size smaller than thefirst image size.
 10. A moving picture recording apparatus for encodinga moving picture through motion-compensated inter-frame prediction andrecording the encoded data in a storage medium, comprising: encodingmodule for encoding an input moving picture at a predetermined framerate and at a predetermined image size; prediction mode control modulefor controlling a prediction mode of said encoding module; andprediction selection module selecting one of a forward prediction frameby prediction using only a past frame, a bidirectional prediction frameby prediction using a past frame and a future frame, and a frame notusing prediction, wherein said prediction selection module selects theframe in accordance with the prediction mode determined by saidprediction mode control module.
 11. A moving picture encoding method ofencoding a moving picture by utilizing motion-compensated inter-frameprediction, comprising: a first encoding step of encoding the movingpicture at a first frame rate and at a first image size; a secondencoding step of encoding the moving picture at a second frame rate andat a second image size; a motion vector calculation step of calculatinga motion vector to be used by said second encoding step from a motionvector to be used by said first encoding step; and a prediction modecontrol step of controlling to make a prediction mode of said firstencoding step be coincident with a prediction mode of said secondencoding step, so that said motion vector calculated by said motionvector calculation step can be used by said second encoding step.
 12. Amoving picture encoding method of encoding a moving picture by utilizingmotion-compensated inter-frame prediction, comprising: a first encodingstep of encoding the moving picture at a first frame rate and at a firstimage size, by performing at least motion compensation and DCT; a secondencoding step of encoding the moving picture at a second frame rate andat a second image size, by extracting data after the motion compensationor DCT by said first encoding step; a motion vector calculation step ofcalculating a motion vector to be used by said second encoding step froma motion vector to be used by said first encoding step; and a predictionmode control step of controlling to make a prediction mode of said firstencoding step be coincident with a prediction mode of said secondencoding step, so that said motion vector calculated by said motionvector calculation step can be used by said second encoding step.
 13. Amoving picture encoding method of encoding a moving picture by utilizingmotion-compensated inter-frame prediction, comprising: a first encodingstep of encoding an input moving picture at a first frame rate and at afirst image size; a second encoding step of extracting picture dataunder encoding by said first encoding step and encoding the picture dataat a second frame rate and at a second image size; a motion vectorcalculation step of calculating a motion vector to be used by saidsecond encoding step from a motion vector to be used by said firstencoding step; and a prediction mode control step of controlling aprediction mode of said first encoding step so as to make the predictionmode of said first encoding step be coincident with a prediction mode ofsaid second encoding step, so that said motion vector calculated by saidmotion vector calculation step can be used by said second encoding step.14. A moving picture recording/reproducing method of recording encodeddata of a moving picture obtained through motion-compensated inter-frameprediction in a storage medium and reproducing the encoded and recordeddata from the storage medium and decoding the encoded data, comprising:a first encoding step of encoding an input moving picture at a firstframe rate and at a first image size; a decoding step of decoding dataencoded by said first encoding step; a re-encoding module of re-encodinga predetermined frame extracted from frames decoded by said decodingstep, at a second frame rate smaller than the first frame rate and at asecond image size smaller than the first image size; a motion vectorcalculation step of calculating a motion vector to be used forprediction by said re-encoding step from a motion vector decoded by saiddecoding step; and a prediction mode control step of controlling aprediction mode of said first encoding step to make the prediction modeof said first encoding step be coincident with a prediction mode of saidre-encoding step, so that said motion vector calculated by said motionvector calculation step can be used by said re-encoding step.
 15. Amoving picture encoding apparatus for encoding a moving picture byutilizing motion-compensated inter-frame prediction, comprising: a firstencoding module for encoding the moving picture at a first frame rateand at a first image size; a second encoding module for encoding themoving picture at a second frame rate and at a second image size; amotion vector calculation module for calculating a second motion vectorto be used by said second encoding module from a first motion vector tobe used by said first encoding module; and a prediction mode controlmodule for controlling in such a manner that a reference frame of thesecond motion vector calculated by said motion vector calculation modulebecomes correspondent with a reference frame of the first motion vectorof said second encoding module.
 16. A moving picture encoding apparatusfor encoding a moving picture by utilizing motion-compensatedinter-frame prediction, comprising: a first encoding module for encodingan input moving picture at a first frame rate and at a first image size;a second encoding module for extracting picture data under encoding bysaid first encoding module and encoding the picture data at a secondframe rate and at a second image size; a motion vector calculationmodule for calculating a second motion vector to be used for predictionby said second encoding module from a first motion vector to be used forprediction by said by said first encoding module; and a prediction modecontrol module for controlling in such a manner that a reference frameof the second motion vector calculated by said motion vector calculationmodule is made coincident with a reference frame of the first motionvector under encoding by said first encoding module and extracted bysaid second encoding module.